O Problema?

Avaliar o quanto uma pessoa compreende um determinado assunto por meio de um teste não é tarefa fácil. Neste quesito alguns fatores precisam ser levados em consideração, como:

A questão precisa conter informação suficiente para a pessoa tomar uma decisão.
A questão precisa abordar um tópico que foi previamente explicado.
Em questões múltipla escolha, como saber se a pessoa apenas tentou adivinhar a resposta?
Questões mais difíceis deveriam pontuar mais?
Como categorizar uma questão como díficil ou fácil?
A pessoa precisa ter tempo hábil para responder todos os quesitos.

No intuito de tentar amenizar alguns desses problemas, tem-se adotado a Teoria de Resposta ao Item. Dessa forma é possível levar em consideração não apenas se a pessoa acertou ou errou a questão, mas também considerar a trajetória dela durante a prova.

Mas o que é a Teria de Resposta ao Item?

Segundo uma conceituação dada pelos autores Dalton, Heliton e Raquel:

"A Teoria de Resposta ao Item é uma metodologia de avaliação educacional que propõe a capacidade de avaliar traços latentes, ou seja, características do indivíduo que não podem ser medidas diretamente. A TRI é uma alternativa ao método clássico onde as notas são dadas baseadas apenas na quantidade de acertos e erros numa prova.

O que esta metodologia sugere são formas de representar a relação entre a probabilidade de um indivíduo dar uma certa resposta a uma questão e seus traços latentes, proficiências ou habilidades na área de conhecimento avaliada."

Aplicação Geral

A TRI possui uma aplicação bem geral, apesar de no Brasil ela ter se tornado mais conhecida por sua utilização no Exame Nacional do Ensino Médio (ENEM), ela é utilizada em outros exames Nacionais (SAEB e SARESP) e Internacionais (PISA, PIAAC, TIMSS). Além disso, a TRI também é utilizada em outras áreas que não seja educação, como:

Psicologia
Pesquisas na saúde
Pesquisas de satisfação

Aplicação ENEM

No escopo da TRI tem-se a existência de diferentes modelos matemáticos, onde cada um recomendado para um tipo de aplicação, no entanto, foge ao escopo desse artigo a explicação matemática de cada um deles, se você tiver interesse em estudar com mais detalhes sobre a teoria envolvida, ao final do artigo serão adicionadas algumas referências.

Dentre os modelos mais comuns utilizados estão:

Dicotômicos:

Rash Models
2PL e 3PL

Politômicos:

Partial Credit Models
Graded Response Model
Rating Scale Model

A nível de esclarecimento, os dicotômicos estão relacionados à questões onde a existe apenas a possibilidade de acertar ou errar. Já nos politômicos as questões possuem uma graduação de intensidade nas possíveis respostas, como por no exemplo abaixo:

Qual o nível de satisfação com a empresa que está trabalhando?

Muito insatisfeito
Insatisfeito
Satisfeito
Muito satisfeito

Conceito

Iremos focar aqui no modelo 3PL, modelo de três parâmetros, pois o mesmo foi utilizado na implementação que realizamos. Nesse modelo matemático são considerados três fatores que estão envolvidos na resolução de um item (entenda a utlização do termo item como questão de uma prova ou simulado).

Params

Modelo 3PL

Apenas a nível de curiosidade, o modelo 3PL é equacionado da seguinte forma:

\[ P(U_{i, j} = 1 | \theta_{j},a_{i},b_{i},c_{i}) = c_{i} + (1 - c_{i})\frac{e^{a_{i}(\theta_{j} - b_{i})}}{1 + e^{a_{i}(\theta_{j} - b_{i})}} \]

Calma, o intuito de apresentar esta fórmula é apenas o de localiza-los onde cada um dos parâmetros é utilizado.

De forma simples, o que essa equação está nos dizendo é a probabilidade de um respondente acertar um item dado todos os parâmetros citados anteriormente e mais a habilidade do indivíduo.

Um ponto que você deve estar se perguntando é: Como eu vou determinar todos esses parâmetros para um item que nunca foi aplicado?

A seguir, no exemplo, nós poderemos ver na prática como isso é feito, mas de antemão o que acontece é uma estimativa baseando-se nas respostas dos próprios respondentes. Veja a imagem abaixo que ilustra melhor o que acontece.

Implementação

Efetuar a estimação dos parâmetros seria impraticável utilizando caneta e papel, dessa forma nós utilizamos uma lib em R chamada MIRT, bem consolidada entre estatísticos usuários da TRI. Ela nos ajudará a encontrar os parâmetros: a, b e c para cada item em nossa prova e a determinar as habilidades dos respondentes.

Para mais detalhes sobre o MIRT~ por favor visite a documentação, é possível encontrar também um artigo que trata sobre a implementação do mesmo.

Pontos para ficar atento

Alguns pontos são extremamente importantes de levar em consideração na hora de realizar uma correção de um simulado pela TRI:

Amostragem

Procure entender se seu simulado terá dados suficientes para estimação dos parâmetros. Essa é uma tarefa difícil pois não possuímos uma fórmula mágica para tomar essa decisão, porém um consenso citado entre diferentes artigos é que por volta de uma amostra de 200 a 250 respondentes para uma prova de 45 items é possível chegar a valores confiáveis dos parâmetros. Quanto mais items, mais respondentes precisamos.

Além disso, como nós utilizamos a própria amostra para estimação dos parâmetros, a mesma precisa ser o mais representativa possível da população que está sendo trabalhada.

Escala utilizada

A escala utilizada pelo ENEM tem um range de aproximadamente 0 até aproximadamente 1000. É possível garantir esse intervalo pois o MEC possui estudo muito rigoroso entre os items e estimativa dos parâmetros. Enquanto uma instituição não faz tal tipo de estudo, é impossível garantir um mínimo e um máximo para o intervalo. No máximo conseguimos adotar uma média e um desvio que desejamos para a escala de habilidade.

Ordem dos items

Em alguns casos, a ordem dos items pode influenciar no score do respondente. Alguns estudos mostram que items localizados mais ao final da prova/simulado tem maior probabilidade de error. Isso acontence por características empíricas do próprio indivíduo que tende a cansar no decorrer do tempo e da prova/simulado.

No entanto, não é um parâmetro muito influenciador no resultado final, ficando a cargo do professor em como distribuir melhor seus items numa prova.

like a normal .. …. ….. …… …. .. .
skewed postive ……… ……. ….. … .. .
skeewd negative . .. … … ….. ….. ……

Análise e Resultados

Aqui iremos trazer uma análise para um case de um simulado realizado por um de nossos clientes. Foram X items e X respondentes no total.

Libs

library(ltm)

## Loading required package: MASS

## Loading required package: msm

## Loading required package: polycor

library(irtoys)

## Loading required package: sm

## Package 'sm', version 2.2-5.6: type help(sm) for summary information

## 
## Attaching package: 'sm'

## The following object is masked from 'package:MASS':
## 
##     muscle

library(mirt)

## Loading required package: stats4

## Loading required package: lattice

## 
## Attaching package: 'mirt'

## The following object is masked from 'package:ltm':
## 
##     Science

library(CTT)

## 
## Attaching package: 'CTT'

## The following object is masked from 'package:polycor':
## 
##     polyserial

library(tidyverse)

## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ──

## ✓ ggplot2 3.3.3     ✓ purrr   0.3.4
## ✓ tibble  3.1.2     ✓ dplyr   1.0.7
## ✓ tidyr   1.1.3     ✓ stringr 1.4.0
## ✓ readr   1.4.0     ✓ forcats 0.5.1

## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()
## x dplyr::select() masks MASS::select()

library(patchwork)

## 
## Attaching package: 'patchwork'

## The following object is masked from 'package:MASS':
## 
##     area

# dados de humanas
day_1_lenguages_humans_science_str <- sprintf("item_%s", seq(1:95))
day_1_lenguages_humans_science_raw_tbl <- read_csv("data/enem_dia1_linguagem_ciencias_humanas.csv")

## 
## ── Column specification ────────────────────────────────────────────────────────
## cols(
##   examId = col_double(),
##   systemUserId = col_double(),
##   results = col_character()
## )

day_1_lenguages_humans_science_tbl <- day_1_lenguages_humans_science_raw_tbl %>% 
  separate(results, into = day_1_lenguages_humans_science_str, sep = ",") %>% 
  rename(student_code = systemUserId,
         activity_code = examId) %>% 
  mutate_all(funs(type.convert(as.integer(.))))

## Warning: `funs()` was deprecated in dplyr 0.8.0.
## Please use a list of either functions or lambdas: 
## 
##   # Simple named list: 
##   list(mean = mean, median = median)
## 
##   # Auto named with `tibble::lst()`: 
##   tibble::lst(mean, median)
## 
##   # Using lambdas
##   list(~ mean(., trim = .2), ~ median(., na.rm = TRUE))

## Warning in type.convert.default(as.integer(activity_code)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(student_code)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_1)): 'as.is' should be specified
## by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_2)): 'as.is' should be specified
## by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_3)): 'as.is' should be specified
## by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_4)): 'as.is' should be specified
## by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_5)): 'as.is' should be specified
## by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_6)): 'as.is' should be specified
## by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_7)): 'as.is' should be specified
## by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_8)): 'as.is' should be specified
## by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_9)): 'as.is' should be specified
## by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_10)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_11)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_12)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_13)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_14)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_15)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_16)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_17)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_18)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_19)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_20)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_21)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_22)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_23)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_24)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_25)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_26)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_27)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_28)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_29)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_30)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_31)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_32)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_33)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_34)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_35)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_36)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_37)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_38)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_39)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_40)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_41)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_42)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_43)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_44)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_45)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_46)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_47)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_48)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_49)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_50)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_51)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_52)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_53)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_54)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_55)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_56)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_57)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_58)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_59)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_60)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_61)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_62)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_63)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_64)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_65)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_66)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_67)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_68)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_69)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_70)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_71)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_72)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_73)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_74)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_75)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_76)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_77)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_78)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_79)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_80)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_81)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_82)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_83)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_84)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_85)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_86)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_87)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_88)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_89)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_90)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_91)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_92)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_93)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_94)): 'as.is' should be
## specified by the caller; using TRUE

## Warning in type.convert.default(as.integer(item_95)): 'as.is' should be
## specified by the caller; using TRUE

day_1_lenguages_humans_science_final_tbl <- day_1_lenguages_humans_science_tbl %>% 
  dplyr::select(starts_with("item"))

# salvando a lista de ids dos estudantes
student_vect <- day_1_lenguages_humans_science_tbl %>% 
  pluck(2)

# salvando o código da atividade
activity_code <- day_1_lenguages_humans_science_tbl %>% 
  pluck(1)

Dessa forma nosso dataframe está pronto para ser utilizado na modelagem. Cada coluna possui um item e cada linha é uma resposta para o item feita por um aluno individual.

A modelagem é feita utilizando o modelo 3PL para estimação dos parâmetros. Passamos os dados, indicamos que é um problema dicotômico e também o tipo de modelo que desejamos.

mymodel <- mirt(
  day_1_lenguages_humans_science_final_tbl, 
  1,          
  itemtype = '3PL')

## 
Iteration: 1, Log-Lik: -129124.231, Max-Change: 9.19090
Iteration: 2, Log-Lik: -125181.819, Max-Change: 2.19863
Iteration: 3, Log-Lik: -124230.158, Max-Change: 0.77046
Iteration: 4, Log-Lik: -123977.920, Max-Change: 0.81716
Iteration: 5, Log-Lik: -123869.386, Max-Change: 0.72797
Iteration: 6, Log-Lik: -123802.401, Max-Change: 0.33642
Iteration: 7, Log-Lik: -123748.961, Max-Change: 0.69037
Iteration: 8, Log-Lik: -123704.599, Max-Change: 0.42171
Iteration: 9, Log-Lik: -123674.692, Max-Change: 0.33249
Iteration: 10, Log-Lik: -123650.310, Max-Change: 0.39556
Iteration: 11, Log-Lik: -123631.051, Max-Change: 0.33967
Iteration: 12, Log-Lik: -123615.000, Max-Change: 0.31484
Iteration: 13, Log-Lik: -123601.390, Max-Change: 0.26525
Iteration: 14, Log-Lik: -123589.559, Max-Change: 0.41111
Iteration: 15, Log-Lik: -123578.859, Max-Change: 1.03047
Iteration: 16, Log-Lik: -123568.489, Max-Change: 0.37158
Iteration: 17, Log-Lik: -123560.284, Max-Change: 0.23295
Iteration: 18, Log-Lik: -123553.208, Max-Change: 0.23079
Iteration: 19, Log-Lik: -123547.061, Max-Change: 0.23912
Iteration: 20, Log-Lik: -123541.731, Max-Change: 0.22271
Iteration: 21, Log-Lik: -123536.967, Max-Change: 0.23010
Iteration: 22, Log-Lik: -123532.671, Max-Change: 0.22692
Iteration: 23, Log-Lik: -123528.731, Max-Change: 0.23828
Iteration: 24, Log-Lik: -123525.212, Max-Change: 0.23224
Iteration: 25, Log-Lik: -123522.407, Max-Change: 0.21556
Iteration: 26, Log-Lik: -123519.263, Max-Change: 0.23528
Iteration: 27, Log-Lik: -123516.549, Max-Change: 0.23424
Iteration: 28, Log-Lik: -123514.441, Max-Change: 0.20337
Iteration: 29, Log-Lik: -123511.871, Max-Change: 0.24136
Iteration: 30, Log-Lik: -123509.730, Max-Change: 0.21344
Iteration: 31, Log-Lik: -123508.123, Max-Change: 0.17434
Iteration: 32, Log-Lik: -123506.001, Max-Change: 0.20501
Iteration: 33, Log-Lik: -123504.244, Max-Change: 0.20290
Iteration: 34, Log-Lik: -123502.816, Max-Change: 0.19488
Iteration: 35, Log-Lik: -123501.145, Max-Change: 0.21593
Iteration: 36, Log-Lik: -123499.704, Max-Change: 0.20784
Iteration: 37, Log-Lik: -123498.483, Max-Change: 0.25419
Iteration: 38, Log-Lik: -123497.073, Max-Change: 0.23151
Iteration: 39, Log-Lik: -123495.839, Max-Change: 0.06842
Iteration: 40, Log-Lik: -123495.361, Max-Change: 0.87858
Iteration: 41, Log-Lik: -123493.258, Max-Change: 0.18511
Iteration: 42, Log-Lik: -123492.124, Max-Change: 0.15992
Iteration: 43, Log-Lik: -123490.996, Max-Change: 0.15133
Iteration: 44, Log-Lik: -123490.130, Max-Change: 0.25444
Iteration: 45, Log-Lik: -123488.909, Max-Change: 0.12500
Iteration: 46, Log-Lik: -123488.134, Max-Change: 0.28884
Iteration: 47, Log-Lik: -123487.077, Max-Change: 0.19466
Iteration: 48, Log-Lik: -123486.298, Max-Change: 0.37892
Iteration: 49, Log-Lik: -123484.648, Max-Change: 0.16275
Iteration: 50, Log-Lik: -123484.097, Max-Change: 2.27892
Iteration: 51, Log-Lik: -123481.977, Max-Change: 0.27295
Iteration: 52, Log-Lik: -123481.831, Max-Change: 0.71121
Iteration: 53, Log-Lik: -123480.836, Max-Change: 0.28026
Iteration: 54, Log-Lik: -123480.271, Max-Change: 0.62417
Iteration: 55, Log-Lik: -123479.871, Max-Change: 1.41328
Iteration: 56, Log-Lik: -123478.618, Max-Change: 0.32942
Iteration: 57, Log-Lik: -123478.083, Max-Change: 0.55018
Iteration: 58, Log-Lik: -123477.642, Max-Change: 0.12207
Iteration: 59, Log-Lik: -123477.096, Max-Change: 0.00537
Iteration: 60, Log-Lik: -123476.881, Max-Change: 0.00494
Iteration: 61, Log-Lik: -123475.994, Max-Change: 0.00461
Iteration: 62, Log-Lik: -123475.856, Max-Change: 0.00448
Iteration: 63, Log-Lik: -123475.728, Max-Change: 0.00429
Iteration: 64, Log-Lik: -123475.070, Max-Change: 0.00350
Iteration: 65, Log-Lik: -123474.994, Max-Change: 0.00313
Iteration: 66, Log-Lik: -123474.924, Max-Change: 0.00249
Iteration: 67, Log-Lik: -123474.698, Max-Change: 0.00421
Iteration: 68, Log-Lik: -123474.637, Max-Change: 0.00213
Iteration: 69, Log-Lik: -123474.592, Max-Change: 0.00220
Iteration: 70, Log-Lik: -123474.370, Max-Change: 0.00216
Iteration: 71, Log-Lik: -123474.342, Max-Change: 0.00195
Iteration: 72, Log-Lik: -123474.316, Max-Change: 0.00187
Iteration: 73, Log-Lik: -123474.206, Max-Change: 0.00142
Iteration: 74, Log-Lik: -123474.188, Max-Change: 0.00243
Iteration: 75, Log-Lik: -123474.174, Max-Change: 0.00151
Iteration: 76, Log-Lik: -123474.143, Max-Change: 0.00223
Iteration: 77, Log-Lik: -123474.130, Max-Change: 0.00128
Iteration: 78, Log-Lik: -123474.115, Max-Change: 0.00169
Iteration: 79, Log-Lik: -123474.068, Max-Change: 0.00127
Iteration: 80, Log-Lik: -123474.056, Max-Change: 0.00174
Iteration: 81, Log-Lik: -123474.048, Max-Change: 0.00117
Iteration: 82, Log-Lik: -123474.027, Max-Change: 0.00179
Iteration: 83, Log-Lik: -123474.021, Max-Change: 0.00097
Iteration: 84, Log-Lik: -123474.013, Max-Change: 0.00134
Iteration: 85, Log-Lik: -123474.003, Max-Change: 0.00091
Iteration: 86, Log-Lik: -123473.996, Max-Change: 0.00130
Iteration: 87, Log-Lik: -123473.991, Max-Change: 0.00094
Iteration: 88, Log-Lik: -123473.980, Max-Change: 0.00043
Iteration: 89, Log-Lik: -123473.978, Max-Change: 0.00035
Iteration: 90, Log-Lik: -123473.976, Max-Change: 0.00031
Iteration: 91, Log-Lik: -123473.968, Max-Change: 0.00026
Iteration: 92, Log-Lik: -123473.967, Max-Change: 0.00026
Iteration: 93, Log-Lik: -123473.965, Max-Change: 0.00025
Iteration: 94, Log-Lik: -123473.959, Max-Change: 0.00025
Iteration: 95, Log-Lik: -123473.958, Max-Change: 0.00024
Iteration: 96, Log-Lik: -123473.957, Max-Change: 0.00023
Iteration: 97, Log-Lik: -123473.952, Max-Change: 0.00021
Iteration: 98, Log-Lik: -123473.952, Max-Change: 0.00021
Iteration: 99, Log-Lik: -123473.951, Max-Change: 0.00021
Iteration: 100, Log-Lik: -123473.947, Max-Change: 0.00095
Iteration: 101, Log-Lik: -123473.944, Max-Change: 0.00087
Iteration: 102, Log-Lik: -123473.942, Max-Change: 0.00083
Iteration: 103, Log-Lik: -123473.932, Max-Change: 0.00031
Iteration: 104, Log-Lik: -123473.932, Max-Change: 0.00086
Iteration: 105, Log-Lik: -123473.931, Max-Change: 0.00026
Iteration: 106, Log-Lik: -123473.931, Max-Change: 0.00118
Iteration: 107, Log-Lik: -123473.931, Max-Change: 0.00033
Iteration: 108, Log-Lik: -123473.930, Max-Change: 0.00088
Iteration: 109, Log-Lik: -123473.930, Max-Change: 0.00034
Iteration: 110, Log-Lik: -123473.929, Max-Change: 0.00102
Iteration: 111, Log-Lik: -123473.928, Max-Change: 0.00032
Iteration: 112, Log-Lik: -123473.928, Max-Change: 0.00028
Iteration: 113, Log-Lik: -123473.928, Max-Change: 0.00076
Iteration: 114, Log-Lik: -123473.927, Max-Change: 0.00018
Iteration: 115, Log-Lik: -123473.927, Max-Change: 0.00083
Iteration: 116, Log-Lik: -123473.927, Max-Change: 0.00034
Iteration: 117, Log-Lik: -123473.926, Max-Change: 0.00079
Iteration: 118, Log-Lik: -123473.927, Max-Change: 0.00040
Iteration: 119, Log-Lik: -123473.926, Max-Change: 0.00016
Iteration: 120, Log-Lik: -123473.925, Max-Change: 0.00059
Iteration: 121, Log-Lik: -123473.925, Max-Change: 0.00024
Iteration: 122, Log-Lik: -123473.925, Max-Change: 0.00062
Iteration: 123, Log-Lik: -123473.924, Max-Change: 0.00015
Iteration: 124, Log-Lik: -123473.924, Max-Change: 0.00070
Iteration: 125, Log-Lik: -123473.924, Max-Change: 0.00029
Iteration: 126, Log-Lik: -123473.924, Max-Change: 0.00067
Iteration: 127, Log-Lik: -123473.924, Max-Change: 0.00035
Iteration: 128, Log-Lik: -123473.923, Max-Change: 0.00068
Iteration: 129, Log-Lik: -123473.923, Max-Change: 0.00031
Iteration: 130, Log-Lik: -123473.923, Max-Change: 0.00026
Iteration: 131, Log-Lik: -123473.922, Max-Change: 0.00060
Iteration: 132, Log-Lik: -123473.922, Max-Change: 0.00018
Iteration: 133, Log-Lik: -123473.922, Max-Change: 0.00015
Iteration: 134, Log-Lik: -123473.922, Max-Change: 0.00057
Iteration: 135, Log-Lik: -123473.922, Max-Change: 0.00073
Iteration: 136, Log-Lik: -123473.921, Max-Change: 0.00018
Iteration: 137, Log-Lik: -123473.921, Max-Change: 0.00057
Iteration: 138, Log-Lik: -123473.921, Max-Change: 0.00083
Iteration: 139, Log-Lik: -123473.921, Max-Change: 0.00020
Iteration: 140, Log-Lik: -123473.921, Max-Change: 0.00057
Iteration: 141, Log-Lik: -123473.920, Max-Change: 0.00018
Iteration: 142, Log-Lik: -123473.920, Max-Change: 0.00077
Iteration: 143, Log-Lik: -123473.920, Max-Change: 0.00026
Iteration: 144, Log-Lik: -123473.920, Max-Change: 0.00056
Iteration: 145, Log-Lik: -123473.920, Max-Change: 0.00040
Iteration: 146, Log-Lik: -123473.919, Max-Change: 0.00017
Iteration: 147, Log-Lik: -123473.919, Max-Change: 0.00056
Iteration: 148, Log-Lik: -123473.919, Max-Change: 0.00018
Iteration: 149, Log-Lik: -123473.919, Max-Change: 0.00056
Iteration: 150, Log-Lik: -123473.919, Max-Change: 0.00016
Iteration: 151, Log-Lik: -123473.919, Max-Change: 0.00070
Iteration: 152, Log-Lik: -123473.918, Max-Change: 0.00024
Iteration: 153, Log-Lik: -123473.918, Max-Change: 0.00056
Iteration: 154, Log-Lik: -123473.918, Max-Change: 0.00035
Iteration: 155, Log-Lik: -123473.918, Max-Change: 0.00075
Iteration: 156, Log-Lik: -123473.918, Max-Change: 0.00026
Iteration: 157, Log-Lik: -123473.918, Max-Change: 0.00022
Iteration: 158, Log-Lik: -123473.917, Max-Change: 0.00055
Iteration: 159, Log-Lik: -123473.917, Max-Change: 0.00019
Iteration: 160, Log-Lik: -123473.917, Max-Change: 0.00016
Iteration: 161, Log-Lik: -123473.917, Max-Change: 0.00055
Iteration: 162, Log-Lik: -123473.917, Max-Change: 0.00055
Iteration: 163, Log-Lik: -123473.917, Max-Change: 0.00020
Iteration: 164, Log-Lik: -123473.917, Max-Change: 0.00055
Iteration: 165, Log-Lik: -123473.916, Max-Change: 0.00014
Iteration: 166, Log-Lik: -123473.916, Max-Change: 0.00012
Iteration: 167, Log-Lik: -123473.916, Max-Change: 0.00055
Iteration: 168, Log-Lik: -123473.916, Max-Change: 0.00055
Iteration: 169, Log-Lik: -123473.916, Max-Change: 0.00017
Iteration: 170, Log-Lik: -123473.916, Max-Change: 0.00055
Iteration: 171, Log-Lik: -123473.916, Max-Change: 0.00015
Iteration: 172, Log-Lik: -123473.916, Max-Change: 0.00012
Iteration: 173, Log-Lik: -123473.915, Max-Change: 0.00054
Iteration: 174, Log-Lik: -123473.915, Max-Change: 0.00054
Iteration: 175, Log-Lik: -123473.915, Max-Change: 0.00018
Iteration: 176, Log-Lik: -123473.915, Max-Change: 0.00054
Iteration: 177, Log-Lik: -123473.915, Max-Change: 0.00012
Iteration: 178, Log-Lik: -123473.915, Max-Change: 0.00011
Iteration: 179, Log-Lik: -123473.915, Max-Change: 0.00054
Iteration: 180, Log-Lik: -123473.915, Max-Change: 0.00054
Iteration: 181, Log-Lik: -123473.914, Max-Change: 0.00016
Iteration: 182, Log-Lik: -123473.914, Max-Change: 0.00054
Iteration: 183, Log-Lik: -123473.914, Max-Change: 0.00014
Iteration: 184, Log-Lik: -123473.914, Max-Change: 0.00011
Iteration: 185, Log-Lik: -123473.914, Max-Change: 0.00054
Iteration: 186, Log-Lik: -123473.914, Max-Change: 0.00053
Iteration: 187, Log-Lik: -123473.914, Max-Change: 0.00017
Iteration: 188, Log-Lik: -123473.914, Max-Change: 0.00053
Iteration: 189, Log-Lik: -123473.913, Max-Change: 0.00012
Iteration: 190, Log-Lik: -123473.913, Max-Change: 0.00011
Iteration: 191, Log-Lik: -123473.913, Max-Change: 0.00053
Iteration: 192, Log-Lik: -123473.913, Max-Change: 0.00053
Iteration: 193, Log-Lik: -123473.913, Max-Change: 0.00015
Iteration: 194, Log-Lik: -123473.913, Max-Change: 0.00053
Iteration: 195, Log-Lik: -123473.913, Max-Change: 0.00013
Iteration: 196, Log-Lik: -123473.913, Max-Change: 0.00011
Iteration: 197, Log-Lik: -123473.913, Max-Change: 0.00053
Iteration: 198, Log-Lik: -123473.913, Max-Change: 0.00053
Iteration: 199, Log-Lik: -123473.912, Max-Change: 0.00016
Iteration: 200, Log-Lik: -123473.912, Max-Change: 0.00053
Iteration: 201, Log-Lik: -123473.912, Max-Change: 0.00011
Iteration: 202, Log-Lik: -123473.912, Max-Change: 0.00010
Iteration: 203, Log-Lik: -123473.912, Max-Change: 0.00052
Iteration: 204, Log-Lik: -123473.912, Max-Change: 0.00052
Iteration: 205, Log-Lik: -123473.912, Max-Change: 0.00014
Iteration: 206, Log-Lik: -123473.912, Max-Change: 0.00052
Iteration: 207, Log-Lik: -123473.912, Max-Change: 0.00012
Iteration: 208, Log-Lik: -123473.911, Max-Change: 0.00010
Iteration: 209, Log-Lik: -123473.911, Max-Change: 0.00052
Iteration: 210, Log-Lik: -123473.911, Max-Change: 0.00052
Iteration: 211, Log-Lik: -123473.911, Max-Change: 0.00015
Iteration: 212, Log-Lik: -123473.911, Max-Change: 0.00052
Iteration: 213, Log-Lik: -123473.911, Max-Change: 0.00012
Iteration: 214, Log-Lik: -123473.911, Max-Change: 0.00010
Iteration: 215, Log-Lik: -123473.911, Max-Change: 0.00052
Iteration: 216, Log-Lik: -123473.911, Max-Change: 0.00052
Iteration: 217, Log-Lik: -123473.911, Max-Change: 0.00014
Iteration: 218, Log-Lik: -123473.910, Max-Change: 0.00051
Iteration: 219, Log-Lik: -123473.910, Max-Change: 0.00011
Iteration: 220, Log-Lik: -123473.910, Max-Change: 0.00010
Iteration: 221, Log-Lik: -123473.910, Max-Change: 0.00051
Iteration: 222, Log-Lik: -123473.910, Max-Change: 0.00051
Iteration: 223, Log-Lik: -123473.910, Max-Change: 0.00014
Iteration: 224, Log-Lik: -123473.910, Max-Change: 0.00051
Iteration: 225, Log-Lik: -123473.910, Max-Change: 0.00012
Iteration: 226, Log-Lik: -123473.910, Max-Change: 0.00010
Iteration: 227, Log-Lik: -123473.910, Max-Change: 0.00051
Iteration: 228, Log-Lik: -123473.909, Max-Change: 0.00051
Iteration: 229, Log-Lik: -123473.909, Max-Change: 0.00015
Iteration: 230, Log-Lik: -123473.909, Max-Change: 0.00051
Iteration: 231, Log-Lik: -123473.909, Max-Change: 0.00011
Iteration: 232, Log-Lik: -123473.909, Max-Change: 0.00010
Iteration: 233, Log-Lik: -123473.909, Max-Change: 0.00051
Iteration: 234, Log-Lik: -123473.909, Max-Change: 0.00050
Iteration: 235, Log-Lik: -123473.909, Max-Change: 0.00014
Iteration: 236, Log-Lik: -123473.909, Max-Change: 0.00050
Iteration: 237, Log-Lik: -123473.909, Max-Change: 0.00012
Iteration: 238, Log-Lik: -123473.909, Max-Change: 0.00010
Iteration: 239, Log-Lik: -123473.909, Max-Change: 0.00050
Iteration: 240, Log-Lik: -123473.908, Max-Change: 0.00050
Iteration: 241, Log-Lik: -123473.908, Max-Change: 0.00015
Iteration: 242, Log-Lik: -123473.908, Max-Change: 0.00050
Iteration: 243, Log-Lik: -123473.908, Max-Change: 0.00011
Iteration: 244, Log-Lik: -123473.908, Max-Change: 0.00010
Iteration: 245, Log-Lik: -123473.908, Max-Change: 0.00050
Iteration: 246, Log-Lik: -123473.908, Max-Change: 0.00050
Iteration: 247, Log-Lik: -123473.908, Max-Change: 0.00014
Iteration: 248, Log-Lik: -123473.908, Max-Change: 0.00050
Iteration: 249, Log-Lik: -123473.908, Max-Change: 0.00012
Iteration: 250, Log-Lik: -123473.908, Max-Change: 0.00010

O output do modelo indica uma convergência do modelo em 250 iterações, com um Max-Change de 0.00010. O que indica que houve uma estabilização para cada estimativa nessa escala.

Dando um print no modelo, nós podemos obter mais parâmetros sobre o modelo, como o optimizer utilizado na simulação e a quantidade estimada de parâmetros, que aqui foi de 285.

Podemos acessar os scores latentes dos items por meio de uma função do pacote do MIRT.

latent_answers <- as.vector(fscores(mymodel))

irt_coefs <- coef(mymodel, simplify = T, IRTpars =T)

Utilizando essa função nós conseguimos transformar os scores para uma nova escala que possui média 500 e desvio de 100.

novo_score <- score.transform(
  latent_answers, mu.new = 500, sd.new = 100,
  normalize = FALSE
)

# data transformations
my_data_column_names <- names(day_1_lenguages_humans_science_final_tbl)

irt_coefs_tbl <- irt_coefs$items %>% 
  as_tibble()

max_n = dim(day_1_lenguages_humans_science_final_tbl[][1])[1]

# student_vect <- 1:max_n
day_1_lenguages_humans_science_final_tbl["student_code"] <- student_vect
irt_coefs_tbl["item"] <- my_data_column_names

mydata_longer <- day_1_lenguages_humans_science_final_tbl %>% 
  pivot_longer(cols = -student_code, names_to = "items", values_to = "count")

score_14_students <- novo_score[1]$new.scores[1:max_n]

scores_tbl <- enframe(score_14_students) %>% 
  rename(student_id = name,
         score = value) %>% 
  mutate(student_id = as_factor(student_vect))

# scores_tbl to export
file_name  <- paste0(as.character(activity_code[1]), "_activiy.csv")
scores_tbl %>% 
  arrange(desc(score)) %>% 
  bind_cols("activity_code" = activity_code) %>% 
  write_csv(file_name)

# adjust color and axis
p1 <- mydata_longer %>% 
  filter(student_code %in% c(1:19)) %>% 
  mutate(items = as_factor(items)) %>% 
  ggplot(aes(x = items,
             y = student_code,
             fill = count == 1)) +
  geom_raster() +
  scale_fill_manual(values = c("#e0e0e0", "orange")) +
  scale_y_discrete(name="Students", limits=c(1:19)) +
  theme_minimal() +
  theme(
    axis.text.x = element_text(angle=45)
  )

## Warning: Continuous limits supplied to discrete scale.
## Did you mean `limits = factor(...)` or `scale_*_continuous()`?

p2 <- scores_tbl %>% 
  mutate(student_id = student_id %>% as_factor()) %>% 
  arrange(desc(score)) %>% 
  top_n(10) %>% 
  ggplot(aes(x = score,
             y = student_id %>% fct_reorder(score))) + 
  geom_bar(stat="identity", width=.7, fill="orange") +
  labs(
    title = "Classfication by score",
    y = "Students IDs",
    x = "Scores"
  ) +
  theme_minimal()

## Selecting by score

p3 <- mydata_longer %>% 
  # filter(student_code %in% c(1:max_n)) %>% 
  group_by(student_code) %>% 
  summarise(total_corrects = sum(count)) %>% 
  ungroup() %>%
  arrange(desc(total_corrects)) %>% 
  top_n(10) %>% 
  mutate(student_code = as_factor(student_code)) %>% 
  ggplot(aes(x = total_corrects,
             y = student_code %>% fct_reorder(total_corrects))) +
  geom_bar(stat="identity", width = .7, fill="orange") +
  labs(
    x = "Quantity of Correct Answer",
    y = "",
    title = "Classification by correct answers"
  ) +
  theme_minimal()

## Selecting by total_corrects

p4 <- irt_coefs_tbl %>% 
  dplyr::select(-c(u, a, g)) %>%
  mutate(item = as_factor(item)) %>% 
  # filter(item != "item_3") %>% 
  ggplot(aes(x = b,
             y = item %>% fct_reorder(b))) +
  geom_bar(stat="identity", width = .7, fill = "orange") +
  theme_minimal() +
  labs(
    x = "b = Difficulty",
    y = "Items",
    title = "Items by order of difficulty"
  )

p2 | p3

IRT Article